Skip to content

feat: remove product config & clean up#790

Open
maltesander wants to merge 18 commits into
mainfrom
refactor/remove-product-config
Open

feat: remove product config & clean up#790
maltesander wants to merge 18 commits into
mainfrom
refactor/remove-product-config

Conversation

@maltesander
Copy link
Copy Markdown
Member

Description

  • remove product config
  • move to config map builder step

Definition of Done Checklist

  • Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
  • Please make sure all these things are done and tick the boxes

Author

  • Changes are OpenShift compatible
  • CRD changes approved
  • CRD documentation for all fields, following the style guide.
  • Helm chart can be installed and deployed operator works
  • Integration tests passed (for non trivial changes)
  • Changes need to be "offline" compatible
  • Links to generated (nightly) docs added
  • Release note snippet added

Reviewer

  • Code contains useful comments
  • Code contains useful logging statements
  • (Integration-)Test cases added
  • Documentation added or updated. Follows the style guide.
  • Changelog updated
  • Cargo.toml only contains references to git tags (not specific commits or branches)

Acceptance

  • Feature Tracker has been updated
  • Proper release label has been added
  • Links to generated (nightly) docs added
  • Release note snippet added
  • Add type/deprecation label & add to the deprecation schedule
  • Add type/experimental label & add to the experimental features tracker

maltesander and others added 16 commits June 2, 2026 18:29
Patches stackable-operator to the operator-rs smooth-operator branch
(provides the v2::config_overrides framework) as the prerequisite for
removing the product-config dependency, mirroring trino-operator.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Copies product-config's writer functions (to_hadoop_xml,
to_java_properties_string) into config::writer, backed directly by the
xml and java-properties crates so output stays byte-identical with the
kuttl ConfigMap snapshots. Removes all product_config::writer usage; the
product-config dependency itself is dropped in a later phase.

Adds unit tests pinning the exact XML/properties on-wire format.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduces the controller/build/properties module structure (mirroring
trino-operator) and a local ConfigFileName enum that replaces the bare
filename string constants as the single source of truth for the
rolegroup ConfigMap keys. Scaffolding for moving per-file rendering out
of the monolithic rolegroup_config_map in the next phase.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Moves rendering of hadoop-policy.xml, ssl-server.xml, ssl-client.xml and
security.properties out of the monolithic rolegroup_config_map match
into dedicated controller/build/properties modules, each with unit tests
pinning the exact output. Logic is copied verbatim (overrides applied
last), so the rendered ConfigMap is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Moves the two large inline config renderings out of rolegroup_config_map
into controller/build/properties/{core_site,hdfs_site}, each with unit
tests driven by a shared MINIMAL_HDFS_YAML fixture. Logic is copied
verbatim (overrides applied last), so the rendered ConfigMap is
unchanged. The match arm in rolegroup_config_map is now a thin dispatch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Moves the recommended networkaddress.cache.ttl=30 and
networkaddress.cache.negative.ttl=0 values out of the product-config
properties.yaml and into security_properties::build directly. While
product-config is still active these arrive identically via the override
map applied last, so the rendered file is unchanged; this makes the
builder self-sufficient ahead of removing product-config validation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replaces the product-config transform/validate steps with native config
merging, mirroring trino-operator:

- validate_cluster no longer takes a ProductConfigManager; it merges the
  typed config fragments (HdfsNodeRole::merged_config) and the per-file
  configOverrides + envOverrides (role <- role group) into the extended
  ValidatedRoleGroupConfig { merged_config, config_overrides, env_overrides }.
- HdfsConfigOverrides now uses typed KeyValueConfigOverrides fields (v2,
  deriving Merge) instead of Options; the per-file builders pull their
  overrides via resolved_overrides and apply them last.
- Operator-injected values previously provided by properties.yaml /
  Configuration impls are now set directly in the builders/validate:
  security.properties DNS cache TTLs, dfs.replication, and the rack-aware
  net.topology / TOPOLOGY_LABELS (namenode).
- Deletes build_role_properties, the Configuration impls, PropertyNameKind
  usage, and the ProductConfigManager plumbing (Ctx, main); the CLI arg is
  kept but ignored.
- Filenames now come from the ConfigFileName enum everywhere (discovery,
  kerberos, jvm), dropping the *_XML / *_PROPERTIES_FILE constants.

The product-config crate dependency is now unused and removed separately.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The product-config crate is no longer used by the operator. Removes it
from the workspace and operator-binary manifests and empties the
config-spec properties.yaml (the schema now lives in the typed CRD and
the per-file builders), mirroring trino-operator. product-config remains
only as a transitive dependency of stackable-operator.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Moves rolegroup_config_map out of hdfs_controller into
controller/build/config_map::build_rolegroup_config_map with its own
Error enum, mirroring trino-operator. The function now takes the
ValidatedCluster and looks up the role group's merged config and
overrides internally instead of receiving them pre-destructured.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mirrors trino-operator: ValidatedCluster now carries a
ValidatedClusterConfig (name, namespace, dfs_replication, https/kerberos/
authentication/authorization flags, rack awareness, OPA authorization)
resolved once during validation. The standalone hdfs_opa_config field is
folded into it.

The build steps no longer take the raw HdfsCluster:
- hdfs_site/core_site builders and build_rolegroup_config_map consume
  &ValidatedClusterConfig / &ValidatedCluster.
- The lower-level HdfsSiteConfigBuilder/CoreSiteConfigBuilder and their
  kerberos security_config impls take primitives (a KerberosConfig struct
  and bools) to keep the config layer free of controller types.

Behavior-preserving: the flags are computed by the same HdfsCluster
predicates as before, just resolved up-front. The discovery path keeps
using the raw HdfsCluster as it does not go through validation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a top-level name: ClusterName field to ValidatedCluster (mirroring
trino-operator), constructed and validated in the validate step. The
per-file builders and build_rolegroup_config_map now take the whole
&ValidatedCluster, reading cluster.name and cluster.cluster_config.

product_version is intentionally not added: unlike Trino (integer
versions parsed to u16), HDFS uses semver product versions, so the image
keeps carrying the version string.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move the pod_refs lookups for namenodes and journalnodes out of
reconcile_hdfs and into validate_cluster, storing the results on
ValidatedCluster. Build steps that already receive the validated
cluster (config_map and hdfs_site) now read the refs from it instead
of taking them as separate parameters.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant